1. Field of the Invention
The present invention relates to an image-shake correcting device, and more particularly relates to an image-shake correcting device that is suitable for application to image-shake correction in cameras, video cameras, and the like.
2. Description of the Related Art
Conventional systems related to a shake correcting function that is provided in apparatuses such as video cameras include, for example, an electronic shake correcting system that directly detects a shake component of an apparatus by an angular velocity sensor, an angular acceleration sensor, or the like, and employs an image pickup element that has more pixels than standard image pickup elements required by television broadcasting systems so that it is possible to extract preselected areas of the standard size of broadcasting systems from within the entire image pickup area of the image pickup element.
FIG. 13 is a schematic diagram showing an image of the image pickup area of an image pickup element according to the electronic shake correcting system of an image pickup apparatus. In FIG. 13, reference numeral 601 designates the entire image pickup area of the image pickup element, and reference numerals 602 to 604 designate areas of the standard size of television broadcasting systems. When performing no shake corrections, the area 603 positioned in the center of the entire image pickup area 601 is extracted by selecting from the three areas 602 to 604, and thus a video image is output. When performing a correction, an area to be selected from the entire image pickup area 601 is shifted, for example, to the area 602 or the area 604 to output a video image such that a shake of the image is removed according to a signal from a shake detecting means, not shown. There is no limitation on the position of an area to be extracted insofar as the area extracted lies within the entire area 601, and the area can even be extracted from an arbitrary position within the entire area 601, which makes it possible to provide images unaffected by hand shakes.
Another example of shake correcting system is an optical shake correcting system that detects a shake component of an apparatus such as a video camera by the shake detecting means described above, offsets the optical axis using lenses that are movable in the perpendicular direction to the optical axis of the lenses, and thus correct shakes.
FIG. 14 is a diagram showing the configuration of groups of lenses of the optical shake correcting system of an image pickup apparatus. In FIG. 14, reference numeral 801 designates a first group of lenses (a fixed lens) fixed to a lens barrel, reference numeral 802 designates a second group of lenses (a zoom lens) that is movable in the direction of the optical axis to vary the power, reference numeral 803 designates a third group of lenses (a shift lens) that is movable in the direction perpendicular to the optical axis to offset the optical axis, reference numeral 804 designates a fourth group of lenses (a focus compensating lens) that moves in the direction of the optical axis to adjust the focus and correct the shift of the focal plane due to the power being varied, and reference numeral 805 designates a charge coupled device (CCD). The shift lens 803 is driven vertically or horizontally according to an output from the shake detecting means described above, and thus carry out shake correction.
Now, a control method for controlling shake correction will be described. Electronic shake correcting control by the area extraction described above and optical shake correcting control by the shift lens described above are mainly performed by typical hand shake correcting control and panning control. First, the typical hand shake correcting control will be described with reference to FIG. 1.
In FIG. 1, reference numeral 101 designates a lens unit, and reference numeral 102 designates a charge coupled device (CCD). A subject image is formed on the CCD 102 by the lens unit 101, and then photoelectrically converted by the CCD 102. The CCD 102 has more pixels than the standard CCDs required by television broadcasting systems (for example, NTSC (National Television System Committee) system). Reference numeral 104 designates a CCD drive circuit for driving the CCD 102. The CCD drive circuit 104 is designed to be able to select lines with respect to the direction V (the number of lines) shown in FIG. 13 described above, from the lines in the entire image pickup area of the CCD 102 in order to extract an area for a final output, wherein the selection is made upon a control instruction from a microcomputer 119 for camera system control described later.
Reference numeral 601 in FIG. 13, mentioned above, designates the entire image size, and reference numerals 602 to 604 designate examples of the standard image size according to television broadcasting systems. In FIG. 13, when the lines starting from ya+1, which is Δya lines below the uppermost line, are effective for example, the Δya lines are read at a high speed, and thereafter the lines from line ya+1 are read out in the same timing as the case of using a CCD of the standard size with respect to a vertical synchronizing signal. Then, the remaining Δyb lines are read out again at a high speed, thus practically extracting lines of the standard image size with respect to the direction V.
Coming back to FIG. 1, reference numeral 103 designates an analog signal processing section that executes a predetermined process on signals obtained from the CCD 102 to generate analogue image pickup signals. Specific examples of the analog signal processing section 103 are a CDS (Co-related Double Sampling) circuit, and an AGC (Automatic Gain Control) circuit. Reference numeral 106 designates a line memory that can store a digital image pickup signal for one line at least, under the control of a memory control circuit 107. Further, pixels can be read out from a predetermined address in the line memory 106, under the control of the memory control circuit 107. Reference numeral 105 designates a camera signal processing section that has a built-in A/D converter and performs processing of digital signals to generate final output video signals. A digital image pickup signal stored in the line memory 106 includes more pixels than the standard image size of the CCD 102, keeping the large number of pixels. The memory control circuit 107 is designed to be able to select a top pixel to be read from the line memory 106, and to read pixels for the standard image size, upon a control instruction from a camera system control microcomputer 119, described in the following.
Reference numeral 119 designates the camera system control microcomputer that performs control of the entire camera system including control of the CCD drive circuit 104, exposure control, white-balance control, variable power lens control, auto focus control, and image stabilizing control. However, in FIG. 1, only a portion of these functions associated with shake correction is shown for brevity. Shake detection is performed with respect to the two axes in the pitch (vertical) direction and the yaw (horizontal) direction. Since the same control is performed for the two axes, FIG. 1 shows the control only for one axis. Reference numeral 111 designates an angular velocity sensor for detecting shakes of the image pickup apparatus. Reference numeral 112 designates a HPF (High Pass Filter) for cutting the DC component of angular velocity signals output from the angular velocity sensor 111. Reference numeral 113 designates an amplifier for amplifying angular velocity signals detected by the angular velocity sensor 111.
Reference numeral 114 designates an A/D converter incorporated into the camera system control microcomputer 119, which converts the angular velocity signals for the two directions described above into digital signals to be used as angular velocity data. Further, a HPF 115 and a phase compensating filter 116 execute predetermined signal processes on this angular velocity data. The angular velocity data passes through a variable HPF 117, which is variable in cutoff frequency (hereinafter referred to as ‘the HPF cutoff frequency’). Then, an integrator 118 generates shake correction signals for the vertical and horizontal directions. Reference numeral 120 designates a correction system controller which transmits the shake correction signals for the vertical direction to the CCD drive circuit 104 and transmits the shake correction signals for the horizontal direction to the memory control circuit 107. As mentioned before, the CCD drive circuit 104 and the memory control circuit 107 change the position for extracting lines from the entire image size of the CCD 102, according to the respective shake correction signals.
Through the above-mentioned series of operations, areas of the standard image size, as designated by reference numerals 602 or 604 for example, can be extracted, as described above, from the entire image size 601 of the CCD 102 with a deviation from the center, as described above, thereby making it possible to correct shakes caused by hand shakes or the like.
When an electronic shake correction by extraction of areas or an optical shake correction by a shift lens, described above, is carried out, a correcting section is provided at a stage subsequent to the zoom lens. In the arrangements of the above two systems, the correction amount for the same amount of shake needs to be corrected correspondingly to a change in the focal distance when the focal distance is changed by a shift of the zoom lens. The reason for this will be explained: Taking an example of a case using a zoom lens with a variable power of 10, if the zoom lens is shifted at its wide end (the end for the longer focal distance) by the same amount as a shift at the telephoto end (the end for the longer focal distance) that requires a shake correction of 0.3 degrees, the resulting shake correction amount at the wide end (the end for the shorter focal distance) is equivalent to 3.0 degrees. A correction method of correcting the correction amount according to the focal distance will be described below, by referring to the case that an electronic shake correction by area extraction is carried out.
Coming back to FIG. 1, reference numeral 121 designates a lens system controller in the camera system control microcomputer 119 that performs control of driving the zoom lens and the focus lens in the lens unit 101, control of autofocus, and the like. The lens system controller 121 controls a zoom motor 109 through a motor driver 108 and shifts the zoom lens to vary the power. Conventionally, a stepping motor has been used as the zoom motor, whereby the position of the zoom lens is detected by drive pulses generated for driving the motor. When a DC motor or a linear motor is used as the zoom motor, the position of the zoom lens can be detected by an encoder which is provided for this purpose. The current focal distance of the zoom lens is recognized from information on the detected position of the zoom lens.
The correction system controller 120 performs correction in response to an output from the integrator 118 and according to the focal distance based on focal distance information detected by the lens system controller 121, and thus calculates a final correction amount. The correction system controller 120 transmits this final correction amount to the CCD drive circuit 104 and the memory control circuit 107, as mentioned before, carry out shake correction. In the above described way, stable hand shake corrections can be achieved irrespective of the focal distance.
Next, panning control will be described. When a photographer carries out panning or tilting of an image pickup apparatus, it is desirable that an image moves as the photographer intends. However, when a normal shake correction is performed during panning, the image does not move due to a shake correction at the beginning of panning, and suddenly starts moving when the shake goes out of the possible shake correction range, giving a sense of discontinuity to the motion of the image. Also, the image gets fixed to a correction limit at the end of panning (getting fixed to a correction limit means a state that the extracting area is fixed to a peripheral edge of the entire image pickup area of the CCD in the electronic shake correction by area extraction, or a state that the shift lens cannot be shifted any more in the lens barrel in the optical shake correction by the shift lens), making shake corrections impossible to carry out.
Panning control is carried out to avoid the above phenomena. Panning control is implemented, for example, such that when an output from the above-mentioned integrator 118 goes out of a predetermined correction range, the cutoff frequency of the variable HPS 117 is changed so that low frequency components are removed, and thus the correction amount is restricted. This panning control makes a shake correction signal indicative of a position close to the center position of the entire image pickup area during panning, which solves the problem described above.
FIG. 15 is a diagram showing the relationship between the output from the integrator and the HPF cutoff frequency in panning control of the image pickup apparatus. In FIG. 15, reference numeral 301 represents a change in the HPF cutoff frequency that corresponds to a change in the output from the integrator. When the output from the integrator exceeds a set value NA, the HPF cutoff frequency is changed as shown in the figure, according to the amount by which the output from the integrator exceeds the set value NA. Further, when the output from the integrator exceeds a set value NB, the gradient of the change in the HPF cutoff frequency is set larger for a greater restriction. In FIG. 15, the ordinate logarithmically represents the HPF cutoff frequency. In this way, the HPF cutoff frequency is changed according to the value of the output from the integrator to restrict the correction amount when panning is performed.
In FIG. 15, reference numeral 303 represents a change in the HPF cutoff frequency at the telephoto end of the zoom lens, and reference numeral 302 represents a change in the HPF cutoff frequency in the vincity of the telephoto end. In the vincity of the telephoto end, the panning speed generally drops. Accordingly, the changing point of the HPF cutoff frequency is shifted, as shown by reference numerals 302 and 303, to the lower side of the output from the integrator for easier transition to panning control, thereby restraining the above described disadvantage during panning.
FIG. 16 is a flowchart showing a panning control process executed by the image pickup apparatus. The panning control process will be described below with reference to FIG. 16. In FIG. 16, in step S1001 it is determined whether the position of the zoom lens is greater than a predetermined value or not. The zoom lens in FIG. 1 described above is shifted by the stepping motor (zoom motor) 109, wherein the zoom lens position is indicated by the number of drive pulses for the stepping motor 109 when it is assumed that the zoom lens position is zero at the wide end. The comparison in step S1001 is for determining whether the zoom lens position is in the vincity of the telephoto end.
FIG. 17 is a diagram showing the relationship between the zoom lens position and the focal distance of the image pickup apparatus. If this relationship is prepared, for example, as table data, a focal distance can easily be obtained by detecting the zoom lens position. If the zoom lens position is greater than the predetermined value in step S1001 described above, in other words, if the zoom lens position is in the vincity of the telephoto end or the focal distance is long, then a predetermined value A and a predetermined value B which are respectively represented by symbol A and symbol B in FIG. 15, described above, are calculated in step S1002. The calculation is carried out as follows.
Provided that set values at the telephoto end of the zoom lens are designated by TA and TB, and set values at a zoom lens position equal to or below the above-mentioned predetermined value by NA and NB, the changing points A and B of the HPF cutoff frequency corresponding to the current zoom lens position are calculated according to the following equations:A=(TA−NA)×(number of all pulses−current number of pulses)/(number of pulses in cutoff frequency changing region)+TA  (1)B=(TB−NB)×(number of all pulses−current number of pulses)/(number of pulses in cutoff frequency changing region)+TB  (2).
If the zoom lens position is equal to or below the predetermined value in step S1001, then the predetermined values A and B are respectively set to NA and NB in step S1003. Next, the absolute value of the output from the integrator is compared with the predetermined value B in step S1004. If the absolute value of the output from the integrator is greater than the predetermined value B in step S1004, then the HPF cutoff frequency is calculated in step S1005.
As the calculation of the HPF cutoff frequency, the HPF cutoff frequency practically used is calculated as index data that corresponds to the HPF cutoff frequency to be set. This index data is specified as table data as shown in FIG. 18A. The table data in FIG. 18A is shown in a graph in FIG. 18B. By determining appropriate index data from the index data shown in FIG. 18A and FIG. 18B, a corresponding HPF cutoff frequency is set. Thus, in step S1005, the above appropriate index data is calculated. In step S1005, k1 represents a gradient from the predetermined value A to the predetermined value B, and k2 represents a gradient for values greater than the predetermined value B in FIG. 15.
If the absolute value of the output from the integrator is equal to or below the predetermined value B in step S1004 described above, then the absolute value of the output from the integrator is compared with the predetermined value A in step S1006. If the absolute value of the output from the integrator is greater than the predetermined value A instep S1006, then index data is calculated in step S1007 in the same manner as in step S1005. If the absolute value of the output from the integrator is equal to or below the predetermined value A instep S1006, then the index data is set to 0 for normal control in step S1008. In step S1009, the HPF cutoff frequency that corresponds to the calculated index data is set. By the operation described above, the HPF cutoff frequency during panning is set and panning control is performed, with the focal distance taken into account.
Although the method of changing the HPF cutoff frequency for panning control has been described above, a similar control is also possible by changing the integration constant of the integrator 118.
However, there is the following problem with the above described prior art. That is, in recent years, lenses with increased magnification have come to be mounted in image pickup apparatuses such as video cameras and hence the focal distance at the telephoto end has been greatly extended. However, the longer the focal distance of the lens, the narrower the correction angle during electronic image stabilizing control. The correction amount that determines the position the area to be extracted from the entire area of the CCD is calculated according to the following equation:Focal distance×tan(correction angle)=correction amount.  (3)
It is to be understood from this expression that the longer the focal distance, the greater the correction amount for the same correction angle.
On the other hand, the maximum correction amount from the center position can be represented by(number of effective pixels of CCD−number of extracted pixels)×unit cell size÷2  (4)
As an example, the case of performing electronic image stabilizing control by extracting an area equivalent to ⅕ inches (290,000 effective pixels) from a ¼ inch CCD having 420,000 effective pixels will be discussed below. The unit cell size of this CCD in the vertical direction is 4.70 μm. With regard to lenses with a focal distance of 4.2 mm at the wide end for example, the correction angles for the vertical direction of a lens with a magnification ratio of 10 and a lens with a magnification ratio of 25 are calculated to be 0.3 degrees and 0.12 degrees, respectively. If the magnification ratio is yet greater, the correction angle is further narrower.
In the electronic shake correction by area extraction, if the correction angle is narrow as described above, the image gets fixed to a correction limit (the state that the extracting area gets fixed to a peripheral edge of the entire image pickup area of the CCD) even during normal control, which is not shake correction or panning, and accordingly the phenomenon that image stabilization is not obtained is apt to occur. There is also a problem during panning that the extracting position immediately shifts to a correction limit even if conventional panning is carried out. To prevent the image from being fixed to the correction limit using the characteristics of the panning control described in the conventional as it is, the relationship between the output from the integrator and the HPF cutoff frequency is required to have a characteristic indicated by a dot-line in FIG. 19, which keeps a panning control state all the time, thus causing a problem that the stabilization characteristic is degraded even upon a slight hand shake.
Also, in the case of performing optical shake correction using a shift lens, if the lens is miniaturized at the sacrifice of the correction angle due to miniaturization of the main body of a video camera, there is also a problem that image stabilization is apt to be ineffective as well.
This gives rise to a disadvantage that images picked up by such an image pickup apparatus, a video camera for example, give a very strange feeling as in the case where no panning control is carried out.